Method and system for predicting personality traits, capabilities and suggested interactions from images of a person

ABSTRACT

The invention relates to a method of predicting personality characteristic from images of a subject person&#39;s face, comprising: a) collecting training images of multiple persons for training propose, the images associated with metadata characteristics of human personality; b) grouping the collected training images into training groups; c) training at least one image-based classifier to predict at least one characteristics of human personality from at least one image of a second person; and d) applying the at least one image-based classifier to at least one image of the subject person for outputting a prediction of at least one human personality characteristic of the subject person.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/700,315 filed on Apr. 30, 2015, now U.S. Pat. No. 10,019,653, whichis a Continuation-In-Part of PCT Patent Application No.PCT/IL2013/050892 having International Filing Date of Oct. 31, 2013,which claims the benefit of U.S. Provisional Patent Application Nos.61/858,686 filed on Jul. 26, 2013 and 61/721,571 filed on Nov. 2, 2012.The contents of the above applications are all incorporated by referenceas if fully set forth herein in their entirety.

FIELD AND BACKGROUND OF THE INVENTION

The present invention relates to the field of machine learning systems.More particularly, the invention relates to a method for predictingpersonality traits based on automated computerized or computer assistedanalysis of that person's body images and in particular face images.

BACKGROUND OF THE INVENTION

Humans judge other humans based on their face images, predictingpersonality traits (such as generosity, reliability), capabilities(intelligence, precision) even guessing professions (a teacher, acare-giver, a lawyer) from a face image alone. Psychological researchhas found a high degree of correlation in such judgments (differentpeople interpreting the same face image in a similar manner). Moreover,psychological research has also found a certain degree of correlationbetween face appearances and ground truth or real-world performance(successful CEO, winning martial arts fighter, etc).

Psychologists, counselors, coaches, therapists gather information onone's personal traits and those of others to analyze and advise oninteractions in the social and business domains. However, it is clearthat different people have different judgment capabilities, somejudgments may be pure prejudice, and in any case it is impractical torely on human judgment to process high-volumes of data in an efficientand repeatable manner.

In the prior art, face image analysis techniques have been provided todetect the emotional state of a person—e.g. anger/happiness/sadness bytracking or recognizing an expression defined by certain deformation ofthe face image as measured for example from the relative distancesbetween facial landmarks, e.g., as disclosed by US Patent applicationNo. 2011/0141258 “emotion recognition method and system thereof”. Incontrast, the present invention measures traits or fixed personalitycharacteristics which do not change over time. Actually, a neutralexpression is preferred, as non-neutral expression, in particular anextreme emotional state, may distort the usual appearance of the personbeing analyzed.

It is an object of the present invention to provide a system which iscapable of predicting personality traits based on automated computerizedor computer assisted analysis of that person's body images and inparticular face images.

It is another an object of the present invention to provide an automatedmethod of selecting personality traits and capabilities that can bepredicted from face images and predicting such traits and capabilitiesfrom one or more face images.

It is yet another object of the present invention to mechanize theprocess of personality analysis and interaction management, usingautomated methods in the field of image analysis, video analysis,machine learning and natural language generation.

Other objects and advantages of the invention will become apparent asthe description proceeds.

SUMMARY OF THE INVENTION

The present invention relates to a method of predicting personalitytraits from at least one image of a subject person body, in particularimages of the person's face, comprising:

-   -   a) Collecting training images of multiple persons for machine        learning training propose in order to identify personality        traits from said images, wherein each of said training images is        associated with metadata characteristics of human personality        traits;    -   b) Grouping said collected training images into training groups        according to said associated metadata, either according to the        same metadata or similar metadata;    -   c) Applying machine learning algorithm(s) on the images in at        least one of said training groups for training at least one        image-based classifier to predict at least one characteristics        of human personality trait from at least one image of a specific        subject person; and    -   d) Applying said at least one image-based classifier to at least        one image of said specific subject person for outputting a        prediction of at least one human personality trait of said        specific subject person.

According to an embodiment of the invention, the personalitycharacteristics and the associated metadata characteristics are selectedfrom the group consisting of: at least one personality trait from a setof human traits, or at least one personal capability from a set ofcapabilities, or at least one behavior from a set of human behaviors.According to an embodiment of the invention, the associated metadata isat least one of the following: profession(researcher/lawyer/coach/psychologist), online behavior (buyer type),endorsements from social network (LinkedIn), crowd source, real-worldbehavior (location, travel, etc.).

According to an embodiment of the invention, the method furthercomprises converting selected face images into a standard, normalizedrepresentation by performing geometric rectification and/orfrontalization on said face images.

According to an embodiment of the invention, the method furthercomprises applying techniques of pose classification in order tofacilitate representation, learning and classification of the images.

According to an embodiment of the invention, the method furthercomprises aligning face images into the nearest reference pose, in thecase where not enough full frontal or side profile images are availablefor personality analysis.

According to an embodiment of the invention, the method furthercomprises linking groups of face landmarks into contours, therebyobtaining another type of face region segmentation, which in particularis useful for the chin area and the ears in side profile view.

According to an embodiment of the invention, the method furthercomprises providing an image descriptors computation module forgenerating multiple image descriptors from the whole face images or fromspecific face parts, during classifier development process to facilitatea specific trait or capability, wherein using said multiple imagedescriptors, an array of classifier modules is able to predict one ormore personality traits/capability, either with or without an associatedmagnitude. According to an aspect of the invention, the method furthercomprises integrating the one or more personality traits/capability withassociated magnitude into a coherent set of personality descriptors,such that whenever a descriptor is manifested in more than one result, aweighting process produces a weighted combination of the individualresults.

According to an embodiment of the invention, the method furthercomprises: a) predicting at least one personality characteristic, fromthe at least one image of the subject person; and combining said atleast one personality characteristic with other personalitycharacteristics or with at least one additional metadata relating tosaid subject person into a composite score associated metadatacharacteristics. For example, the additional metadata can be demographicdata. The composite score can be obtained from at least one face-derivedpersonality trait/capability/behavior with optional metadata, by methodof weighing.

According to an embodiment of the invention, the method furthercomprises searching/ranking individuals by face-based personalitytraits/capabilities that include the steps of:

-   -   a) Collecting at least one face image for each of said        individuals;    -   b) Predicting at least one personal trait from a set of human        traits, or at least one personal capability from a set of        capabilities, or at least one behavior from a set of human        behaviors, from said at least one face image for each of said        individuals;    -   c) Combining said at least one personal trait from a set of        human traits, or at least one personal capability from a set of        capabilities, or at least one behavior from a set of human        behaviors, with other trait/capability/behavior or at least one        additional metadata relating to each of said individuals into a        composite score; and    -   d) Ordering said individuals based on said composite score and        selecting at least one individual based on said ordering.

According to an embodiment of the invention, the method of weighingincludes at least one weight computed using training data and machinelearning techniques. In one aspect, the method of weighing includes atleast one weight assigned manually. In another aspect, the method ofweighing includes at least one weight computed using training data andmachine learning techniques.

According to an embodiment of the invention, the method furthercomprises generating description of multiple personality characteristicsby applying a plurality of image-based classifiers to one or images ofthe subject person, wherein said personality description is obtained bya face-based personality analysis module.

According to an embodiment of the invention, the method furthercomprises implementing the face-based personality analysis module inmultimedia systems or applications adapted to interact with one or morepersons in real-time, such that face images of person(s) can be capturedfrom an imaging module associated with said multimedia system orapplication, in order to be analyzed by said module, either in real-timeor off-line. For example, the multimedia systems or applications can beselected from the group consisting of: video chat, conferences call,wearable computers, portable computer based devices, desktop computerbased systems, CRM, set-top boxes, smartphones, gaming consoles, videocameras, smartphones and other mobile devices.

According to an embodiment of the invention, the process of theface-based personality analysis module may further comprise one or moreadditional tasks such as: searching for additional images of the subjectperson by using a name search engine that can be augmented by facerecognition by face recognition and analyzing said additional images toenhance the accuracy of predicted personality traits or capabilities.For example, analyzing the content of textual data during theinteraction, converting audio signals of verbal communication during theinteraction to written communications, either to be presented during theinteraction and/or to be analyzed for content and meaning, analyzingaudio signals of verbal communication by voice-based analyzer forobtaining personality/situation/emotion cues, and generating interactionrecommendations—generic and content-based, according to the descriptionof personality characteristics, such that interaction analysis and thegenerated recommendations are integrated within the applications. Theface-based personality analysis module can be configured to presentpersonality characteristics of the person(s) during the interaction withsaid person(s). In some embodiments, the face-based personality analysismodule presents personality characteristics of the person(s) during theinteraction with said person(s). The method may further compriseanalyzing information during the interaction from plurality of contentsources, including content of textual data either written data orconverted audio signals of verbal communication during the interactionand integrating such information with predicted personality traits.

According to an embodiment of the invention, the method furthercomprises classifying the type of the subject person according to one ormore predicted personality characteristics, thereby allowingfacilitating personal advertising by providing adaptive message to saidsubject person according to said classifications.

According to another aspect the present invention relates to acomputer-readable medium that stores instructions executable by one ormore processing devices to perform a method for predicting personalitycharacteristic from at least one image of a person, comprising:

-   -   a) Instructions for collecting images of multiple persons with        associated metadata characteristic of human personality;    -   b) Instructions for grouping said collected images into training        groups according to said metadata;    -   c) Instructions for training at least one image-based classifier        to predict at least one characteristic of human personality from        said at least one image of said person; and    -   d) Instructions for applying said at least one image-based        classifier to said at least one image of said person and        accordingly outputting a prediction of at least one        characteristic of human personality.

The present invention permits the characterization of one's personalitybased on automated computerized or computer-assisted analysis of thatperson's body images.

According to the present invention, analysis output can be in concisetext form, as list of traits with magnitude for each such trait.

Additionally, the present invention allows generating rich-textpersonality descriptions by combining Natural Language Generation (NLG)with the above-mentioned analysis output.

In a specific embodiment of the present invention, the output style istailored to the reader's personality (a sensitive person, one with senseof humor, etc.).

Once one's personality traits and capabilities are available(pre-computed/in real-time), the present invention allows managingperson to person interaction and/or machine-person interaction using oneor more of the following interaction management techniques:

-   -   Analyzing the personality of 2 or more people and further        analyzing the interactions between them (example: couple,        parent-child, employer-employee, etc.).    -   Providing best practices recommendation for improved interaction        and communication of said 2 or more people (for example,        marriage consulting).    -   Analyzing the personality of a designated person and suggesting        preferred practices of approaching and further interacting with        that person to the person performing the interaction (the “user”        which may be a sales person, customer support person, emergency        services person).    -   Resolving personal and interpersonal communication problems, by        suggesting best practices for the analyzed person to communicate        with other persons in social and business environments.    -   Recommending best matches and best practices in matching,        sexing, dating other persons, based on compatible        characteristics and traits    -   Combining the personality traits of the designated person with        pre-computed personality traits of said “user” to further        improve/focus said preferred practices.    -   Analyzing the personality of a designated person and evaluating        the suitability of that person to a job or function or purpose        [filtering candidates, interaction best practices for        interview]. Said suitability can be defined as a correlation        measure between the personality traits of the designated person        and the preferred personality characteristics for said        job/function/purpose, where purpose may be one or more of the        following: sexing/dating/marriage.    -   Identifying strong and weak points of opponents in debates,        contests, reality programs, games, gambling, sports and further        devising winning strategies and best practices to optimize the        user positioning and outcomes of such interactions.    -   Providing a search tool/filter/engine based on personality        traits, in various domains, locations, or for people which may:        -   possess one or more pre-defined traits such as “kind”,            “intelligent”, “analytic”;        -   have matching/compatible/opposing traits to me/someone I            know/a famous individual (movie star/sport champion or            celebrity);        -   exhibit a certain online/real-world behavior:            criminal/terrorist, aggressive, gambling, buying, investing,            early adopter in fashion or technology;        -   have a significant life-time value as a customer.

The personality traits used by said search tool are primarily derivedfrom face images according to the present invention. However, additionaltags/metadata can be used by the search tool to facilitate or focussearch including personality traits obtained by other means,non-personality metadata (such a location, age, and gender), web-basedendorsements (e.g., LinkedIn endorsements), etc.

As a specific example, the task of matching young candidates to aresearcher position, when a significant track record which may reflectdesired traits and capabilities is not available for the candidate, andtherefore the candidate's future performance must be predicted.

Current searching for the right candidate relies on reading resumes,interviewing candidates, conducting reference calls and observing thecandidate behavior in “group dynamics” observed by psychologists.

Now the present invention facilitates finding the candidate by:

-   -   Gathering face images of all candidates: asking candidates to        furnish such images (preferably in full frontal or side profile        views), or using a face image gathering process as described in        the present invention (see FIG. 2).    -   Computing one or more personality traits from said face images,        per the present invention. Alternatively only key traits as        required for the specific job are extracted. Each computed trait        is associated with a value/magnitude of the trait.    -   Correlating the set of required traits (and values) and the set        of computed traits (and values) to produce a matching score        between the candidate and the position. As not all traits are        equally important the correlation will be weighted, based on        relative importance of weight associated with the specific        trait.    -   Once the set of candidates is filtered as described above and        several of these candidates are summoned for an interview or        another decision/persuasion process, the system will generate a        proposed interaction with each of these candidates, to get the        best result from such further process. For that purpose the        system according to the present invention may suggest strong and        weak points of the candidate to elaborate on during interview or        for further investigation.

Alternatively, according to a different embodiment of the presentinvention, a more “integral” approach may be selected, by training aclassifier to predict the membership of a person to the “researchers”group, using a large training set of researchers in the domain ofinterest and a comparable training set of persons from the generalpopulation.

In a different set of embodiments of the present invention, the “user”is not a human but a machine, computer or other automated/mechanicalmeans such as: Information kiosk, Vending machine, Robotic systems,Gaming console/software, Gambling software, Customer support/CRMsoftware, Any product/software user interface (UI/UX), Medical prognosisand interaction with patients, Smartphone/computerized intelligence orinteractions like personal assistance applications using voice and/orspeech recognition, Sales to businesses and to consumers, Securityintelligence, detecting fraud, suspects, Banking, ATM whereidentification of dishonest traits will elevate the security measures,requires further identifying details, will trigger an alert, etc.,Advertising to people based on the consumer traits and characteristics,Machine/robot interaction with humans based on the technology, Smart TV

According to a further embodiment, personality traits are combined withauxiliary data captured during the interaction, to improve the accuracyor relevance of said auxiliary data and to improve the interactionmanagement.

Auxiliary data may be captured in real-time or recorded and usedafter-the-fact for interaction analysis.

According to one such embodiment, said Auxiliary data comprise one'swritten/spoken messages as obtained from textual interaction(SMS/e-mail) or verbal interaction (using speech recognition).

According to one further embodiment, one's personality traits arefurther integrated/combined with automated body language/expressionanalysis/voice and speech analysis thus obtaining dynamic, real-timeinformation to augment the personality traits.

According to another embodiment, personality traits derived from bodyimages according to the present invention are combined with furtherpersonality cues obtained from handwriting recognition to providebroader/more accurate analysis.

According to an embodiment of the present invention, the method furthercomprises:

-   -   a) capturing one or more images of a human by a mechanical or        virtual artificial agent;    -   b) predicting personality traits of said human by using at least        one image-based classifier, thereby enabling to personalize an        interaction of said artificial agent with said human.

According to an embodiment of the present invention, the metadatacontains one or more performance scores, thereby enabling to generateleads in the field of performance based advertising, such that the priceper lead or other measure of value is related to a performance measureachievable from at least one of said leads.

In another aspect, the present invention relates to a computer-readablemedium that stores instructions executable by one or more processingdevices to perform a method for predicting personality traits from atleast one image of a person, comprising:

-   -   a) Instructions for collecting images of multiple persons with        associated metadata characteristic of human personality traits        and/or capabilities;    -   b) Instructions for grouping said collected images into training        groups according to said metadata;    -   c) Instructions for training at least one image-based classifier        to predict at least one characteristic of human personality from        said at least one image of said person; and

Instructions for applying said at least one image-based classifier tosaid at least one image of said person and accordingly outputting aprediction of at least one characteristic of human personality.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

In the drawings:

FIG. 1 schematically illustrates a system for extraction of personalitytraits from face images, according to an embodiment of the presentinvention;

FIG. 2 schematically illustrates a face gathering module of the system,according to an embodiment of the invention;

FIG. 3 schematically illustrates a high-level framework for personaltrait, capability or behavior learning/training and thenprediction/classification, according to an embodiment of the invention;

FIG. 4 schematically illustrates a high-level framework for personaltrait, capability or behavior learning/training and thenprediction/classification with respect to face-based demographicclassification, according to an embodiment of the invention;

FIG. 5 schematically illustrates an exemplary key points on an image ofa person's face;

FIGS. 6 and 7 schematically illustrate a threshold-based decisionstrategies in 2-class and 3-class classification problems, according toembodiments of the invention;

FIG. 8 illustrates applicable schemes for the personalityfeature/descriptor extraction, according to an embodiment of the presentinvention;

FIG. 9 shows a presentation of the system's outputs as a result of anidentified personality traits and magnitudes;

FIG. 10 depicts an exemplary video chat/conference system enhanced bypersonality analysis and interaction recommendations, according to anembodiment of the invention;

FIG. 11 depicts an exemplary integration of personality analysis in awearable computing device arrangement, according to an embodiment of theinvention;

FIG. 12 shows a specific example of an early adopter buyer typecharacterized by traits;

FIGS. 13 and 14 depict a score computation methods from biometricinformation such as face images, according to an embodiment of thepresent invention;

FIG. 15 depicts a face harvesting process, according to an embodiment ofthe present invention;

FIG. 16 depicts an embodiment for high-performance lead generationsystem, according to an embodiment of the present invention; and

FIG. 17 schematically illustrates an implementation of a roboticservice, according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Throughout this description the invention is described for the specificcase of face images as such images are widely available and/or can beeasily captured and provide a majority of personality traits. However,it is clear from the description of the method how to extend theinvention to other images that include: head images, hand images, bodyimages. Each body part can produce a personality profile alone orcombined with other body parts into a richer, more comprehensiveanalysis.

Reference will now be made to several embodiments of the presentinvention, examples of which are illustrated in the accompanyingfigures. Wherever practicable similar or like reference numbers may beused in the figures and may indicate similar or like functionality. Thefigures depict embodiments of the present invention for purposes ofillustration only. One skilled in the art will readily recognize fromthe following description that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles of the invention described herein.

The method of the present invention teaches the use of machine visionand machine learning techniques to infer personality traits, classmembership and personal capabilities traits from face images, thusenabling rapid, automated, accurate and repeatable analysis of massvolumes of humans. The present invention allows automaticallyidentifying/classifying/describing/quantifying such traits using onlyface images of the analyzed person. According to the present invention,analysis output can be in concise text form, as list of traits withmagnitude for each such trait.

The following discussions intended to provide a brief, generaldescription of a suitable computing environment in which the method ofthe invention may be implemented. While the invention will be describedin the general context of program modules that execute in conjunctionwith an application program that runs on an operating system on apersonal computer, those skilled in the art will recognize that theinvention may also be implemented in combination with other programmodules.

FIG. 1 schematically illustrates a system 10 for extraction ofpersonality traits from face images, according to an embodiment of thepresent invention. The system 10 comprises a server 100, a plurality of(face) images sources 101 and at least one terminal device 102. In thisembodiment, server 100 includes the following modules: a face gatheringmodule 110, a face landmark detection module 120, a face normalizationmodule 130, a face region segmentation module 140, image descriptorscomputation module 150, an array of classifier modules 160, weightingand integration module 170 and a profile generation and recommendationengine module 180. The server 100, the sources 101 and the terminaldevices 102 may communicate via a data network such as the Internet.

Module 110 gathers multiple face images of the same persons frommultiple sources 101 which may include:

-   -   Live capture from stand-alone/integrated camera, as individual        images or as a video image sequence (a recorded video clip);    -   Hand held device such as smartphone, tablet, camera;    -   Wearable camera with optional display such as Google Glass;    -   Video-enhanced Internet application;    -   Video conference system;    -   Images furnished by the designated person (a job applicant/as        part of registration);    -   A social network (such as Facebook, LinkedIn and the like);    -   Online photo album management and sharing services such as        Picasa;    -   Search engines such as Google Images;

Face Landmark Detection module 120 performs geometric rectificationand/or frontalization of face images, e.g., by searching for specificfacial key points that are useful in verifying the face pose and laterconverting the face image into a standard, normalized representationthat may represent a frontal pose, and may have an essentially neutralexpression. Such key points customarily include the eyes, eye corners,eyebrows, the nose top, the mouth, the chin, etc. as indicated by theblack dots on the face image in FIG. 5. Higher level of detail mayinclude dozens of such points which may be used as an image descriptorfor the learning and prediction steps.

Face images are captured in multiple poses and expressions. Tofacilitate representation, learning and classification we assume thatall images are either full frontal or side profile images. In mostsituations, a large number of available images will allow selecting suchimages for training and prediction, where such selection can be manualor automatic, using prior art techniques of pose classification.

In the case where not enough full frontal or side profile images areavailable for personality analysis, face normalization module 130,aligns the face image into the nearest reference pose. Suchnormalization is necessary in order to derived normalized metrics foridentification of personality traits and/or to compare the face regionimages with database images, said comparison being in direct form(image-to-image) or by extracting and comparing/classifying features orimage descriptors. Reference pose may include the full frontal pose, theside profile pose and other poses. The face normalization can beobtained by implementing known techniques such as disclosed by thepublication of X. Chai et al., “Pose Normalization for Robust FaceRecognition Based on Statistical Affine Transformation, IEEE Pacific-RimConference on Multimedia, Singapore 2003.

Face segmentation module 140 uses normalized face images and thelocation of face landmarks to designate face regions to be used for faceparts analysis 150.

Face segmentation may use multiple methods. For example, by gatheringskin tone statistics as the dominant color component of the face,regions that deviate from skin color may be detected—such as the lipsand the eyes.

By linking groups of face landmarks into contours, another type of faceregion segmentation is obtained. This is useful for the chin area, forthe ears in side profile view, etc.

Alternatively, when image descriptors used for learning and predictionare extracted from the entire face images, Face image segmentation 150is used to mask out background details that may affect the accuracy ofthe classifiers, optionally including the subject hair.

Image descriptors computation module 150 generates multiple imagedescriptors from the whole face images or from specific face parts thathave been shown, during classifier development to provide betteraccuracy for a specific trait or capability. Said descriptors which aredetailed below should be characteristic of face structure, immune todisturbances such as illumination effect, of relatively lowdimensionality and of course suitable for learning and accurateprediction.

Using said face part or whole face descriptors, an array of classifiermodules 160 predict one or more personality traits/capability withassociated magnitude. A collection of such personality traits/capabilityis indicated by numeral 161.

Module 170 integrates the collection 161 into a coherent set ofpersonality descriptors. Whenever a descriptor is manifested in morethan one result, a weighting process produces a weighted combination(for example a weighted average) of the individual results.

In a specific embodiment, weighs are assigned manually, based on HR bestpractices, sociological research, etc. In another specific embodiment,weighs are learned automatically from a training group of successfulindividuals (successful from real-world metadata, according to crowdsource, etc.) using machine learning tools such as Support VectorMachine (SVM), boosting, neural networks, decision trees and others.

Module 180 translates the identified traits and their values/magnitudesinto a suitable presentation of the personality profile and/orinteraction recommendation. An exemplary output is described in furtherdetails with respect to FIG. 9 hereinafter.

Module 180 also produces certain output to the user, regarding thequality of the input pictures, the amount of information extracted andfurther instructions. For example, if the gathered images are fullfrontal image only, than some traits requiring side profile images willbe missing. The system will prompt the user with messages such as:

-   -   Personality profile 75% done.    -   Please add side profile images

FIG. 2 describes the face gathering module 110 in further details,according to an embodiment of the invention. Image search engines 210such as Google Images generate multiple search results upon a namesearch. Certain filters in the search engine allow returning only imageswith faces, only images larger than a certain size, etc.

Similarly, face images of the designated person may be collected throughsocial networks (e.g., “Facebook”), online photo album (e.g., “Picasa”),optionally incorporating human or machine tagging of said photos.

Search results may include face of different people, face images withunsuitable quality, pose, etc. Such false matches or low-quality imagesmay offset the results of personality analysis per the presentinvention.

According to the present invention an automated process select onlyappropriate images from the search results.

Face recognition technology may be used to ensure that all selectedimages depict the same, correct person. In the case that one or moreimages of the target person have been tagged manually, a facerecognition engine 220 will pick face pictures of the same person fromthe face results. Alternatively, in the lack of a tagged key image, anautomatic grouping process based on face similarity will group imagesubsets where each subset corresponds to a single person. The largestsimilarity group shall belong to the person of interest with higherprobability. Alternatively, the largest groups shall be inspected by ahuman who will select the appropriate subset. An automatic groupingprocess can be implemented by using techniques such as disclosed by X.Zhang, Y. Gao, Face recognition across pose: A review, PatternRecognition 42 (2009) 2876-2896.

These similarity-filtered search results are further analyzed to selectimages of high quality, of neutral expression and of appropriate pose(for example full frontal images and side profile images). Face qualitymodule 230 uses quality metrics such as face size, face image sharpnessto select face select images of good quality. Other measures may includeface visibility or level of occlusion—such as from glasses, hair style.Such analysis can be implemented by using techniques such as disclosedby Y. Wong et al., Patch-based Probabilistic Image Quality Assessmentfor Face Selection and Improved Video-based Face Recognition, CVPR 2011.

A possible embodiment of step 230 can use the landmark detection processusing the number of detectable landmarks as a face quality metric aswell as for pose detection and subsequent alignment.

Face expression analysis modules 240 further selects face images ofneutral expression, in order to avoid biased results of face personalityanalysis due to extreme expressions. Such expression analysis can beimplemented by using techniques such as disclosed by B. Fasel and J.Luettin, Automatic Facial Expression Analysis: A Survey (1999), PatternRecognition, 36, pp. 259-275, 1999.

Pose filtering step 250 selects and classifies images of the preferredfull frontal or side profile pose. In the case that not enough imagesare available with these poses initially, pose rectification step 260 isemployed (see description with respect to module 130 hereinabove).

Social networks such as Facebook are a common source for harvestinghuman faces. Many Facebook images are provided with tags. As oneexample, the tagging is provided as a list of image rectangles labeledby ID of users, where such rectangles ideally intersect with therespective users' face regions. FIG. 15 depicts a face harvestingprocess according to an embodiment of the present invention.

Given the User ID (UID), selected (e.g., by minimum file size) or allimages are downloaded from the user's online photo album. Face Detectionmodule 1510 extracts all detectable faces in each image, associating abounding rectangle with each detected face. Then, the photo tagginginformation, provided as a rectangle per UID is correlated by TagCorrelation 1520 to find intersection with a detected face. Once alluser photos are extracted, they undergo a selection process as describedin FIG. 2 to provide the best image(s) to the training process.Specifically, steps 230, 240, 250 and optionally alignment step 260 areused.

When the source of face images is a video image sequence, the steps ofquality filtering 230, expression filtering 240 and pose filtering 250are conducted on multiple images from the sequence to select goodimages. Still, the selected images may be highly redundant, as ifsequence dynamics are slow. In such a case, key-frame selection methodas known in prior art may be used to reduce the number of face images.Alternatively, one can use face similarity metrics to detect redundancyand select a reduced number of representative face images.

When multiple images of same person are suitable for analysis, suchmultiple images can be combined to increase the accuracy of saidanalysis. As one example of combining multiple images, the images areanalyzed independently, producing a set of trait values for each image.Then a statistical process such as majority voting or other smoothing orfiltering process is applied to produce a robust statistical estimate ofsaid trait value.

FIG. 3 schematically illustrates a high-level framework for personaltrait, capability or behavior learning/training and thenprediction/classification, according to an embodiment of the invention.In the description that follows we present specific embodiments of thesteps in that framework.

As one specific example, we construct a classifier to differentiatebetween near eyebrows and far eyebrows. Near eyebrows are usuallyassociated with a person comfortable with close range interaction,either verbal or physical, for example a sales person, a martial artsfighter, etc.

A first step is collecting examples of face images of persons with neareyebrows and a comparable collecting process for persons with fareyebrows.

Following face detection, alignment and cropping step 310, we have 2sets of face images normalized at least according to the followingparameters: image size, face image size, face location and orientation.The training process typically requires hundreds to low thousands ofimages of each set for the purpose of training and testing the methodsdescribed below. Each of these images is associated with metadatacharacteristics of human personality.

Before applying step 320 “Feature Extraction”, we must select/design adescriptor that will be invariant to illumination, skin color, and fineimage structures and so on. There are several possibilities. Some of thebest known descriptors for image-based classifiers are SIFT/LBP/HOG.

SIFT=Scale Invariant Feature Transform extracts from an image acollection of feature vectors, each of which is invariant to imagetranslation, scaling, and rotation, partially invariant to illuminationchanges and robust to local geometric distortion. Lowe, David G. (1999),“Object recognition from local scale-invariant features”, Proceedings ofthe International Conference on Computer Vision 2, pp. 1150-1157.

LBP=Local Binary Pattern

To compute the LBP descriptor, divide the examined window into cells(e.g. 16×16 pixels for each cell). Then, for each pixel in a cell,compare the pixel to each of its 8 neighbors (on its left-top,left-middle, left-bottom, right-top, etc.). Follow the pixels along acircle, i.e. clockwise or counter-clockwise. Where the center pixel'svalue is greater than the neighbor's value, write “1”. Otherwise, write“0”. This gives an 8-digit binary number (which is usually converted todecimal for convenience). Compute the histogram, over the cell, of thefrequency of each “number” occurring (i.e., each combination of whichpixels are smaller and which are greater than the center). Optionallynormalize the histogram. Then, concatenate normalized histograms of allcells to the feature vector for the window. For example, see T. Ojala,M. Pietikäinen, and D. Harwood (1996), “A Comparative Study of TextureMeasures with Classification Based on Feature Distributions”, PatternRecognition, vol. 29, pp. 51-59.

HOG=Histogram of Oriented Gradients (HOG) counts occurrences of gradientorientation in localized portions of an image. “An HOG-LBP HumanDetector with Partial Occlusion Handling”, Xiaoyu Wang, Tony X. Han,Shuicheng Yan, ICCV 2009.

According to specific embodiment, the SIFT descriptor is used andspecific details are available below. Note that the descriptor iscomposed of edges in patches from all parts of the face—thus the samedescriptor can be used to classify traits associated with differentparts of the face such as lips, eyes, and nose. The learning algorithm(e.g., SVM in a specific embodiment) will weigh the relevant coordinatesfor a specific trait according to relevant patches that best describeit.

In a specific embodiment, according to the SIFT descriptor, we extract150 image “windows” from the normalized face image which undergoes aprocess of spatial gradient computation. Each image window is thendivided into 4*4 sub-windows and the gradient content of each window isrepresented by a histogram of gradient orientation quantized to 8directions. So, initially, the descriptor dimension is 150*4*4*8=19200.Such a high dimension can make the classification computation difficult,so we reduce each of the 150 vectors of dimension 128 to a vector ofdimension 10 using Principal Component Analysis (PCA), to obtain a totaldimension of 1500 per face image. Alternatively, a different imagedescriptor such as LBP or HOG might be used.

In a different embodiment, a face landmark detection algorithm such asthe one commercially available from Luxand Inc. (seehttp://www.luxand.com) in Software Development Kit (SDK) form to detectdozens of tagged face landmark from a feature. If for example, 50landmarks are detectable, then with normalized (x,y) value for eachlandmark, a descriptor of dimension 100 is obtained.

After the feature extraction, at step 330, the images are grouped intotraining groups according to their associated metadata characteristic ofhuman personality.

We now describe step 340, applying a machine learning algorithm to thetraining images and corresponding descriptors, labeled according tometadata (e.g. low eyebrows vs. high eyebrows) to train a classifier350. The description will be based on the technique of Support VectorMachines (SVM), but different “machine learning” algorithms can also beused, such as Cortes, Corinna; and Vapnik, Vladimir N.; “Support-VectorNetworks”, Machine Learning, 20, 1995.

Using SVM the system tries to find a classifier in 1500d that willseparate between, for example, the faces labeled as having closeeyebrows and faces labeled as having far eyebrows (from the eye). Themost common form of SVM: the binary classifier is designed for a 2 groupcase. For the case of personality/capability/behavior analysis thesystem often needs 3 levels of magnitude: low/balanced/high. There areseveral options for that:

-   -   Compute a threshold “t” such that:        -   Classifier output >=t is assigned the highest magnitude (3)        -   Classifier output between −t and t is assigned the middle            magnitude (2)        -   Classifier output below −t is assigned the lowest magnitude            (1).    -   Multi-class SVM (one-versus-one) using voting of several binary        SVM. To identify 3 magnitudes for a certain trait we need to        train 3 binary classifiers:        -   a classifier that will separate between images with            magnitude 1 and 2;        -   a classifier that will separate between images with            magnitude 3 and 2; and        -   a classifier that will separate between images with            magnitude 1 and 3.

Now, for a new image we will activate all 3 classifiers and check whichmagnitude got maximum votes.

-   -   Observing that the task at hand it is not an ordinary        multi-class problem as we have order between the classes:        Magnitude 1 is lower than magnitude 2 and this one is lower than        magnitude 3.

Therefore, we will use ranking—SVM-rank is trained by giving it pairs(of images) and tag which one is bigger (according to a certain trait wewant to work on). In this specific example, the classifier is trained todetect a “longer than” relation between 2 images. Thus, given 2 faceimages the classifier shall decide which has longer distance from theeyebrows to the eyes.

-   -   Another option is using Regression—deriving a function that will        map the descriptor to the level of magnitude of the trait. SVM        is applicable to the case of regression, maintaining all the        main features that characterize the maximal margin algorithm: a        non-linear function is learned by a linear learning machine in a        kernel-induced feature space, e.g., see Cristianini and        Shawe-Taylor (2000).    -   Kernel method: mapping the descriptor from a general set S into        an inner product space V (equipped with its natural norm),        without ever having to compute the mapping explicitly, hoping        that the observations will gain meaningful linear structure        in V. The term inner product space refers herein to a vector        space, in linear algebra, with an additional structure called an        inner product. This additional structure associates each pair of        vectors in the space with a scalar quantity known as the inner        product of the vectors. Possible mapping include non-linear        function such as:    -   polynomial: (gamma*u′*v+coef0)̂degree    -   radial basis function: exp(−gamma*|u−v|̂2)    -   sigmoid: tanh(gamma*u′*v+coef0)

FIG. 4 schematically illustrates a high-level framework for personaltrait, capability or behavior learning/training and thenprediction/classification with respect to face-based demographicclassification, according to an embodiment of the invention. In thedescription that follows we present specific embodiments of the steps inthat framework.

Face-based demographic classification (gender, age, ethnicity) is knownin prior art. The idea is that improved personalitytrait/behavior/capability classification may benefit from demographicsegmentation. For example, the system collects images of maleresearchers and female researchers, doing the same with a control group,which in this context may comprise of people that are known not to beresearchers. Now we train a classifier for male researcher and one forfemale researcher (and of course verify during development that webenefit from such segmentation).

For training, the source for demographics data may be non-face-baseddemographic metadata (e.g., social network such as from Facebookprofile) or face-based demographic data (either human-tagged or machinetagged) as extracted and classified by blocks 375 and 380. The facedetection, alignment and cropping (block 360) and the personality facefeature extraction are similar in their functionally to blocks 310 and320 as described with respect to FIG. 3 hereinabove. Grouping accordingto demographics and personality/behavior/trait metadata (block 370) isdone in a similar manner to block 330 as described with respect to FIG.3 hereinabove.

During prediction, the subject face image undergoes face-baseddemographic classification (blocks 375 and 380). Alternatively,non-face-based demographic data is retrieved (if available). Then thedemographic-specific personality trait/behavior/capability classifier isapplied by array of classifiers (block 390) and according to the machinelearning algorithm (block 385).

Computing the Threshold

According to the embodiments described above, a threshold is used toimplement a 3-class decision. In a different embodiment of a 2-classcase a threshold is used when it is not mandatory to classify all facesand it is important to reduce the classification error. Consider forexample tagging specific members of a group (e.g., a loyalty club) asearly adopters of new products and technology, for the purpose ofmailing to them a product sample, an invitation to an event etc. Whensuch marketing method requires an investment in every single prospect,it is crucial to spend the budget wisely. Assume that the entireaudience is a very large. Hence, a possible approach may be to assigncertain members of the audience a “Don't Know” tag, even if 50% (in anextreme case) of the audience are not classified at all—provided thatthe remaining audience is classified at 90% accuracy. To implement thatstrategy we select a threshold t such that:

-   -   For a classifier output>(+t) we assign class 1 (see FIGS. 6 and        7);    -   For a classifier output<(−t) we assign class 2 (see FIGS. 6 and        7); and    -   For classifier outputs between (−t) and (+t) the system produces        a “Don't Know” undecided output (see FIG. 7).

The threshold can be calculated as the minimum result of the classifieron the training positive examples. It also can be asymmetric thresholdabove and below the classifier. In that case we can also calculate theresult on the negative example and look for the closest one to themargin. It can also be any value between the closest and the furthestresults of the training—depends on the amount of “Don't Know” we want toallow.

FIGS. 6 and 7 depict the threshold-based decision strategies in 2-classand 3-class classification problems, according to aforementionedthreshold selection strategy. The SVM classifier is indicated by numeral710.

According to FIG. 6 the decision strategy works as follows:

In step 711 the system check if the classifier output S is smaller thanthe threshold (−t). If yes, then class 1 is assigned to the classifier.If no, in step 712, the system checks if the classifier output S isbigger than the threshold (+t). If yes, then class 3 is assigned to theclassifier, and if no then class 3 is assigned to the classifier.

According to FIG. 7 the decision strategy works as follows:

In step 711 the system check if the classifier output S is smaller thanthe threshold (−t). If yes, then class 1 is assigned to the classifier.If no, in step 712, the system checks if the classifier output S isbigger than the threshold (+t). If yes, then class 2 is assigned to theclassifier, and if the classifier outputs a value between (−t) and (+t)then “Don't Know” is produced.

To achieve high classification precision, a few hundred training samplesare required for each trait and magnitude. For certain uncommon traitsit may be difficult to collect an appropriate amount of trainingsamples. According to a specific embodiment, a 2-step process initiallylearns from a relatively small number of samples, forming a low/mediumaccuracy classifier. Then using the initial classifier examples can becollected, manually clean the mistakes and strengthen the model bytraining a classifier from a much higher number of samples.

FIG. 8 illustrates applicable schemes for the personalityfeature/descriptor extraction, according to an embodiment of the presentinvention.

According to one specific embodiment, (830, 835) face regions or theentire face are classified by appearance based methods that use no apriori knowledge on the data present in the image. Instead, they trythrough statistical analysis of the available dataset (either an imageor image characteristics database) to extract the different variationmodes of the database. Several appearance-based descriptors includeSIFT, LBP or HOG transforms. In another embodiment, Haar features areused to represent a face. The Haar wavelets are a natural set basisfunctions which encode difference in average intensities betweenadjacent regions. These features have been found useful in facedetection and can be easily computed with the Integral image using thedifferences of the sum of the pixel values within neighboringrectangular regions. As this set of over-complete, the potential numberof such features is huge and a bootstrapping process may be required toselect the features. For example, see “A general framework for objectdetection Computer Vision”, 1998. Sixth International Conference on(1998), pp. 555-562 by C. P. Papageorgiou, M. Oren, T. Poggio.

Other face regions associated with personality traits are betterdescribed with contour-based methods (810, 815). For example, traitsassociated with the jaw/chin and with the nose side profile contour. Thecontour may be represented by Fourier Descriptors. For example, see C.S. Lin and C. L. Hwang, “New Forms of Shape Invariants from EllipticFourier Descriptors”, Pattern Recognition, vol. 20, no. 5, pp. 535-545,1987. Additional descriptors are suited for shape and contour: S.Belongie and J. Malik (2000). “Matching with Shape Contexts”, IEEEWorkshop on Content based Access of Image and Video Libraries(CBAIVL-2000). Chamfer distance: [D. M. Gavrila and V. Philomin,Real-time Object Detection using Distance Transforms, Proc. of IEEEIntelligent Vehicles Symposium, pp. 274-279, Stuttgart, Germany, 1998.Edges fragments—Incremental learning of object detectors using a visualshape alphabet Oplet, Pinz, Zisserman (2006).

Other face structure descriptors can be describes by simple normalizedmetrics (820, 825). For example, consider the height variation of theinner corners of the eyes. Once the face image is rotated to canonicposition, the height difference between the eye corners is detected andnormalized by the distance between these corners.

Once features or descriptors are associated with each personality trait,a trait classifier is assigned to each descriptor/trait. The classifierdepends on the specific representation of said features or descriptors.The training process is usually done offline, and therefore memory orcomputation time requirement are relaxed. According to the bottom partof FIG. 3, during prediction, an unknown input image (or set of imagesdepicting the same individual) undergoes similar detection, alignmentand cropping step 310 as in the learning phase. The normalized image ispassed through a feature extractor stage, generating features/imagedescriptors that are then passed to the classifier. According to oneembodiment of the present invention, is classifier is SVM.

The output of the classifier may be a tag or a label which is aprediction of the metadata (with optional magnitude) from the domain ofmetadata supplied with the training images in the learning stage. In oneexample the input metadata during learning tags the membership of aperson to a specific group (“researcher”, “Poker player”, “earlyadopter”, “economical buyer”) and the classifier output provides anindication of whether the personality traits and capabilities of theindividual are compatible with those of the specific group. The queryphase, for all traits that use the same descriptor and are classified bylinear classifier, can be calculated at once very fast by one matrixmultiplication (combined from all linear classifiers).

Personality and Health Profiling

According to one embodiment of the invention, the metadata contains atleast one trait from a list of personality traits as known inpsychology. As a specific example, the Big Five personality traits arefive broad domains or dimensions of personality that include: openness,conscientiousness, extraversion, agreeableness and neuroticism. Suchmetadata for the training group may be obtained through psychologicalquestionnaires and interviews as well as self and peer ratings[Goldberg, Lewis R. “The development of markers for the Big-Five factorstructure.” Psychological assessment 4.1 (1992): 26.]

Given a large training database of individuals, tagged with Big-5 traitvalues, a classifier for each of these trait values is constructedaccording to the present invention. Consider for example theextraversion trait and assume that through the psychologicalquestionnaires, each individual of the training is assigned a numericalvalue between 0 and 100 where low scores indicate high-level ofintroversion and high scores denote a high level of extroversion.

Our training group will consist of a group of individuals with a lowscore (say 0-30) and a comparable group with high scores (70-100). Faceimages are then collected and analyzed according to the presentinvention, resulting in a face-based extraversion trait classifier.Afterwards, the classifier can be applied to the general population andpredict low/high extraversion trait values using face images only,without the cost and effort of having the general population fillquestionnaires or conduct interviews/peer ratings.

According to an embodiment of the present invention, a psychologicalprofile is constructed from multiple classifiers, yielding for examplethe complete big-5 profile of an individual using his/her face imageonly.

According to another embodiment of the invention, the metadata containsone or more elements of a health profile and classifiers are generatedto predict one's health elements—to the extent that such elements areproven to be predictive of such elements.

Crowd Source

According to a further embodiment of the present invention, crowd sourceis used to improve the system's analysis capabilities. According to oneembodiment, a person performs analysis of himself or of a person he knowwell. When the description is presented to that person he is asked toagree/disagree (could be level from 1-5) to each trait. Then, thespecific face region(s) associated with each trait (with high/lowagreement scores) are used as positive/negative examples for thetraining process.

Occasional such inputs may be biased or erroneous, however assuming thatmost such inputs will be correct/authentic, the classification systemwill exhibit a “learning” curve.

Generating Rich Descriptions

Natural Language Generation (NLG) is the process of converting acomputer based representation into a natural language representation. InNLG the system needs to make decisions about how to put a concept intowords. More complex NLG systems dynamically create texts to meet acommunicative goal. This can be done using either explicit models oflanguage (e.g., grammars) and the domain, or using statistical modelsderived by analyzing human-written texts.

According to the present invention, personality analysis and interactionrecommendation are converted from their computed attributes and valuesinto a verbal description using NLG module.

FIG. 9 shows a presentation of the system's outputs as a result of theidentified personality traits and magnitudes. An output/NLG module(s)1000 is used to generate the system's output.

In accordance with the embodiments of the present invention, personalitytraits and capabilities identified and assigned values/magnitudes,trigger the generation and output (display/file/message) of short formdescriptions (denoted as HIGHLIGHT and indicated by numeral 1001). Forexample, the HIGHLIGHT descriptions may include terms such as clear andcoherent verbal ability, articulate, talkative, etc.

Furthermore, NLG engine generates 1000 a:

-   -   Detailed description of the specific trait—denoted as DETAILED        and indicated by numeral 1002. The detailed description may        include expression such as “you verbally express yourself in a        fast, simple and flowing way”, “you are inclined to embellish        your conversation with fancy adjectives”, etc.    -   Best practice of interaction (3rd person advice)—denoted as        INTERACTION and indicated by numeral 1003. For example, the        INTERACTION may provide information such as: “The best way to        address this type of person in conversation is to ask him very        short questions”.    -   Recommendations to 1st person—denoted as First Person and        indicated by numeral 1004. The recommendation may include one or        more advantages (e.g., “you are able to engage in long        conversations”), challenges (e.g., “a long conversation tends to        exhaust the receiving end, since you can go an on endlessly”),        and recommendations/advices (e.g., “pay close attention to body        language so you could see if the person you are speaking to has        lost interest in what you are saying”).

All the above will be better understood through the followingillustrative and non-limitative examples. Specific embodiments of thepresent invention allow its usage to facilitate interaction in multipledomains and applications. Such applications may include: Robotics, SalesImprovement, Coaching, Smart CRM, Negotiation Tool, Personaladvertising, HR recruitment, Teaching Aid, Criminology Analysis,Personal Gaming, Meeting Intelligence, Career Guidance, etc.

We describe below several embodiments of machine-assisted interactionsin the social and business domains. It should be straightforward forsomeone skilled in the art to extend the described embodiments foradditional domains and interactions.

Video Chat/Conference

Video chats/calls/conference are a very popular tool for remote socialand business interaction with people. Multiple face images of theparticipants are naturally available but most participants lack theknowledge to “read” the information embedded in these images and/orinduce the desired interactions with the other participants.

FIG. 10 depicts an exemplary video chat/conference system enhanced bypersonality analysis and interaction recommendations. All 3 levels ofinteraction are analyzed by the system of the present invention.

A face-based personality analysis module generates description ofpersonality trait as described in the present invention. Face images arecaptured from video during the call itself and optionally stored forlater interactions. Additionally, a name search engine, optionallyaugmented by face recognition, as described in FIG. 2 hereinabove,brings in additional pictures of the same person, with better quality,controlled pose, illumination etc.

The personality analysis module according to the present invention 1020presents personality traits of the person/persons participating in thechat/video conference.

As most video chat/conference systems provide means to type textualmessages, to all participants or to selected participants, a text-basedinteraction analysis module 1030 generates analyzes the textualinteraction during the chat/conference.

A speech-based interaction analysis module 1040 includes a speechrecognition module as known in prior art, converting any verbalcommunication to written communications, to be presented on the screenand also analyzed for content and meaning. Additionally, the audiosignal will be analyzed by voice-based analyzer forpersonality/situation/emotion cues such as loudness, stress, pausesbefore and after certain words, etc.

An interaction expert 1060 system combines cues from all 3 interactionanalysis module into interaction recommendations—generic andcontent-based. Moreover, the expert system guides the specific modulesabove in what to analyze and what to present. For example, the topic ofthe current discussion, as identified from speech recognition and textanalysis, will select what personality traits and interactionrecommendation to present at any given moment.

From the user's point of view, interaction analysis & recommendationsdisplay 1110 is integrated within the video chat/conference application.

CRM Integration

According to one embodiment, the present invention facilitates CustomerRelationships Management (CRM). Current CRM systems use technology toorganize, automate, and synchronize business processes—principally salesactivities, but also those for marketing, customer service, andtechnical support, in order to find, attract, and win new clients,service and retain those the company already has, entice former clientsto return, and reduce the costs of marketing and client service.

From the aspect of utilizing customer information to facilitate theabove processes, Current CRM systems are limited to informationfurnished by the customer or prospective client, through customerregistration, past transactions or inquiries, and optionally informationmanually entered by company personnel documenting past interactions.

According to the present invention, a personality profile is generatedfrom client images as collected from the Internet or provided throughvideo conference interactions. Then, when a specific interaction withthe customer is planned, the system would present recommendations/bestpractices for that interaction.

This would assist remote customer support, sales and service personnelto better interaction with a prospective/current/dissatisfied customer,as a standalone application or integrated with intelligent onlineengagement solutions using chat, such as those provided by LivePerson.

Personal Advertising

According to another embodiment, the present invention facilitatesPersonal Advertising. The infrastructure for adaptive messaging doesexist. For example TV set-top boxes may display different messages atdifferent households at the same time-slots. Also personal advertisingfit seamlessly into web content and any non-linearly consumed content.Such messages may be promoting different products of different vendors,different products of same vendor or same product, with different waysof presenting the messages to different audiences.

The challenge remains how to select the best message to show at anygiven time.

Prior art techniques collect information on user preferences by trackingweb surfing history, TV viewing habits and any information furnished bythe user. For anonymous applications such as out-of-home advertising,non-cooperative techniques identify external characteristics such as agegroup and gender. In multi-user environment, such as a TV in the livingroom, user identification methods based on face recognition or onidentification of viewing habits.

However, all these techniques gather and utilize externalcharacteristics and fail to infer “internal” user traits and hencepreferences. According to the present invention, face images of saiduser are analyzed to predict his or her personality traits andrecommended interaction. Then a message is chosen from a collection ofmessages or customized in a manner that increases the relevance of themessage to the viewer's personality.

According to a specific embodiment of the present inventionusers/viewers are classified into one or more types, such as “buyertypes”, for example a type from the list “early adopter”, “me too”,“economical” and “sensitive”. In certain embodiments it may be valuableto designate a “primary” buyer type but also a “secondary” buyer typewhich yields better selectivity/targeting.

In one embodiment, the user behavior is tracked and used to provideadditional information to the initial classification. So for example ifone “early adopter” buyer shows more interest in new experiences andanother “early adopter” buyer shows more interest in new technologies,such “behavioral” information may be added to the initial “personality”buyer type/profile and used for better conversion down the road.

Furthermore, once the system collects enough examples for each of the“behavioral” sub-groups of “early adopter”, one may apply trainingalgorithm from the field of pattern recognition or machine learning, asdescribed in the present invention and fine-tune personality types.

In an anonymous situation, such as out-of-home displays, user faceimages are captured by a connected camera and analyzed per the presentinvention.

When the user can be identified, previously captured face images can beprocessed as described above. Alternatively, previously generatedpersonality descriptors can be retrieved from a user database.

Integration with Wearable Computing Device

According to a further embodiment of the present invention, a wearableimage capture device such as Google Glass is worn by the user whocaptures images of persons of interest in the physical environment ofthe user. The device is preset to one or more of applications such asselecting and approaching a tentative client for product sales orpicking up a partner.

Once the device identifies that the user focuses on a certain “target”person, as can be defined by keeping that person's image in the centerof the device field of view for a minimum preset duration, the devicecaptures face images of the target person. The device then selects atleast one image for personality analysis. Generated traits are presentedin concise form on the display. Optionally, recommendations forinteractions are presented. Such recommendation may be sales/engagementtactics in a shopping venue (“mall”), opening or introduction phrases ina party or a bar, etc.

The above can be implemented with any mobile computing device withintegrated camera, such as smartphone, tablet, iPod, etc. One canperform the complete analysis in the device, or compute the descriptorsand send them to a remote server for the actual classification. Ineither case no image or other personally identifying information is sentoutside the device, thus preserving privacy.

FIG. 11 depicts a possible integration of personality analysis in such awearable camera/computing device arrangement 1100. When the personalityanalysis mode is activated, the device 1100 detects the person ofinterest—for example occupying the center of the field of view (asindicated by the dotted square 1101). Then after a short duration, whena one or more image of sufficient quality have been captured and thepersonality analysis is complete, the system will present a short(bullet-form) summary of that person traits, as well as recommendedinteraction guidelines as indicated by numeral 1102.

Robots

Robots are designed to address certain aspects of marketing and customercare, among other tasks. Robot interaction can be as simple as greetingcustomers in one of several languages [e.g., see the “Toshiba's humanoidretail robot is ready to greet you” at the following URL addresshttp://www.engadget.com/2015/04/20/toshiba-aiko-chihira-robot/]

Throughout this description the term “robot” is used to indicate amechanical or virtual artificial agent, usually an electro-mechanicalmachine that is guided by a computer program or electronic circuitry. Inthe present invention we refer to multiple embodiments of robots fromvending machine, information kiosks, virtual online assistants, tophysical robots, with speech capabilities and optionally motion, gestureand expression capabilities.

It is clear that a uniform interaction is not as effective as apersonalized interaction, where the needs of the human being served areidentified and the message or the entire interaction is tailored tosuite the human's personality.

It is both technically and socially preferred that such personalizationis conducted without accessing the user's connected devices oridentifying the user otherwise. In short, anonymous personalization by arobot is preferred.

According to one embodiment of the present invention, as shown in FIG.17, a robot may capture images of the human through a connected cameraor by other image capturing means, predicts personality traits (forexample, Big-5 traits) of the human by using Face-Based Big-5Personality Classifier 1710 and personalizes the interaction with thehuman to best communicate/serve/protect/sell to that human.

Said personalization of interaction layer to match the human personalitytraits may consist of: at least one of personalizedmessage/answer/dialog and personalized behavior: e.g.quiet/dynamic/friendly, as selected by Robotic Message/BehaviorSelector/Modifier 1720 personalized appearance: gender, age, face shapeand facial feature, body size and shape.

In particular, a virtual (screen display/hologram) or physical avatar(robot), whose face/body will be modified/tailored/selected to thespecific served human, by Robotic Appearance Selector 1730 to get betterinteractions, results, save time, in specific applications such as:

-   -   Vending machines/Automatic Teller Machines (ATM), which will        sell or serve humans/people;    -   Caring machines/robots, which will advise, or serve in        medical/pharmaceutical/sport training;    -   In education, machines/robots, which will teach/educate/train or        advise students;    -   Financial/insurance advisors;    -   Robot drivers/sailors/pilots/tourist guides/guards shall        increase human confidence and usage level by adapting their        appearance and interaction to the human personality;    -   Personal aids, assistants, house-keepers, baby-sitters, adult        helpers, virtual friends shall increase human confidence and        value by adapting their appearance and interaction to the human        personality.

In a specific embodiment, said virtual (screen display/hologram) orphysical avatar (robot), will be modified/tailored/selected/shapedaccording to the specific purpose. For example, if the desired purposeis convincing the user to select a specific service, from say aninsurance company, the robot's face will be selected/shaped to have alarge width to height ratio, which has been shown in psychologicalstudies to be associated with an authoritative personality.

According to another embodiment of the present invention, saidpersonality-adapted appearance shall comprise screen color/background,font shape, size and color. In another embodiment of the presentinvention, robot/virtual aid voice will be selected/modified accordingto the human personality by modifying audio level, voice pitch(male/female) and speed.

Online vs. Offline Analysis

The present invention allows capturing personality traits from faceimages in real-time, using images captured by an integrated/attachedcamera. This has the benefit of immediate response and improvedinteraction without any identifying information regarding the user andalso in situations where the user is “random”—such as out-of-homeadvertising/kiosks. Additionally, previously captured personality traitscan be retrieved from a local/remote database and used duringinteractions with a known/identifiable user. This is the benefit ofmanaging interactions when images are not currently available or notavailable at the required quality.

GUI/UX

According to a further embodiment of the present invention, userpersonality traits are used to facilitate man-machine interaction, bymodifying the content and/or appearances of presented information, basedon such traits. In a simple example, the graphical design of a web pagewill be altered based on user personality to facilitate navigation anddecision making. As a more specific example, a commerce web-page willhighlight the practical advantage of a product for one user, thecost-saving element for another and a design/appearance (“cool factor”)element for yet another user.

As another example, the appearance of the displayed information will bemodified, using flexible User Interface (UI) or User Experience (UX)elements—cold vs. warm colors, font style and size, general layout ofthe display, etc. Such personality-adaptive appearance may cause theuser to spend more time in that screen/web-page and increase exposure topresented information.

Gaming Consoles

Current gaming consoles include a camera as a standard/optional devicefor gesture recognition and other natural user interface applications.Example: Microsoft Kinect. According to a further embodiment of thepresent invention, user face images are captured by the camera, andidentified personality traits are used to enhance the gaming experience.Example: Enhancing first-person shooter games by controlling scene andenemies appearances to suit the player will result in better conversionrates, more playing time, better response to embedded commercialmessages, etc.

Performance Based Advertising

According to another embodiment of the invention, the metadata containsone or more performance scores and such embodiment may be used togenerate leads in the field of performance based advertising.

[“Performance Based Pricing Models in Online Advertising: Cost-Per-Clickvs.Cost-Per-Action”—http://faculty.som.yale.edu/JiwoongShin/Downloads/workingpapers/PerformanceBasedPricingModels.pdf]

In this field, the price per lead or other measure of value is relatedto a performance measure achievable from the lead. Such measure mayinclude the lead's Lifetime Value (LTV), a desired behaviour (viralbehaviour in online gaming), etc. Typically, the customer has aninstalled base of users that achieve or surpass the desired performancelevel. Also, the customer may have a control group comprising of usersthat fail to achieve the desired performance level, or another suitablecontrol group.

FIG. 16 depicts an embodiment for high-performance lead generationsystem, according to an embodiment of the present invention. Theembodiment shall be described for a performance measure which is acustomer Lifetime Value (LTV or CLTV), but is applicable to anyperformance measure or another figure of merit that is desired to beoptimized with respect to a selected group of individuals. In thisembodiment it is assumed that the current user base contains largeenough (typically hundreds) groups with high LTV and low LTV, asdepicted by data 1610 and 1620.

It is further assumed that all users in the specific embodiment arerepresented and are accessible via their UID (User ID).

Given groups 1610 and 1620, a social network face harvesting module 1630(as described in FIG. 15) converts the harvested faces into a set ofaligned and cropped face image which are in turn converted to FaceFeatures/Descriptors by module 1640.

According to the present invention, the inputs of machine learningalgorithms 1650 comprise not only of the respective descriptors butalso:

Target Group Prevalence compared with control group: for example it maybe a high TLV user is 10 time less common that a low TLV user.

The allowed undecided/Don't Know rate. By allowing keeping a largerpercentage of the scanned population undecided, one can increase theaccuracy of the Predicted High TLV leads generated by classifier 1670.To produce the leads, an external user ID database 1660 is harvested andinput to the Classifier 1670.

System Architecture

The present invention is usable in both consumer and corporateapplications (B2B, B2C, C2C), for both in-home and out-of-home, bothembedded in a device or distributed to a local server, a remote serveror cloud processing resources. To support a wide variety ofapplications, possible implementations of the present invention mayinclude one or more of the following modalities:

-   -   PC-based: in form of software pre-installed or downloaded to        one's computer.    -   Remote-server: all analysis steps 110-180 in FIG. 1 are        conducted on a remote server. Face images captured by the user        are uploaded to the server which responds with personality        traits and interactions guidelines.    -   Cloud-based: an online file storage web service running on a        cloud infrastructure such as Amazon S3.    -   Distributed: face detection and pose, quality and expression        filtering per step 120-150 in FIG. 1 is done on the client side.        This is most useful for video-based applications on a smartphone        or a tablet, where client-side processing reduces computation        and communications overload on the server. An additional merit        is that of privacy where the image descriptors generated by        module 150 and sent over a communication networks cannot be        reconstructed as an image or other personally identifiable data.    -   Application Programming Interface (API) which allows embedding        the personality trait identification into 3^(rd) party client        products/applications, while relying on remote server processing        as described above.    -   Software Development Kit (SDK) which provides PC-based        processing or remote server processing to be included in a        3^(rd) party application.    -   Embedded as DSP/processor software, IP core provided as part of        a chip or a chip design allows placing the entire knowledge on        the device for independent operation and fast response.

Behavioral Training of Trait Classifiers

The present invention teaches how to compute trait values from faceimages, based on prior knowledge that relates face/face part appearanceto such trait values. In a specific example, an early adopter buyer typemay characterized by the traits as shown in the tables 1210 and 1220 inFIG. 12.

According to an additional embodiment of the present invention it ispossible to train classifiers based on behaviorally tagged face images.In the example above, we can tag certain people as early adopters basedon their online behavior, questionnaires, membership in a certain group,etc.

In this specific example, suppose that we have a second group of usersthat can be tagged as economical buyer type, again based onlinebehavior, purchase history, questionnaires, and membership in a certaingroup.

Without breaking either behavior into specific traits, one may train aclassifier to distinguish between these two groups. This has thefollowing advantage: Faster, More Accurate, May be used to define newbehavioral groups without the need to break them into based traits.

According to the present invention, it is possible to predict themembership of a person to a certain group of people (e.g., researchers).

Collecting images of faces of people who known to be researchers (fromuniversities' websites) and images of people who known to benon-researchers—such as models, basketball players and actors. Then wenormalize and calculate an appearance descriptor (SIFT) as describedabove and look for the best SVM classifier (kernel, parameters andthreshold) to separate between those two groups.

Alternatively, perform multi-trait analysis for each face image,resulting in a vector of magnitudes, which serves as the descriptor.That way, we can separate between the wanted group and the otherpopulation and we could find a representative of the group—thus knowwhich traits and magnitude are typical and describe this group—that canalso be expand to classification of several groups.

Note: If this group is infrequent in the normal population otherlearning methods should be used such as boosting or nu-svc.

Training with Performance Monitoring Tools

According to a specific embodiment of the present invention it isrequired to group users into specific categories based on their onlinebehavior and then train specific classifiers to predict online behaviorbased on that person's face images only. Specific examples of currenttools include: Google Analytics, user experience management tools fromCompuware, etc.

According to the present invention, performance monitoring tools areused to collect user behavior data for users and identify behavioralpatterns of the users. In parallel, face images of these users arecollected via mechanisms such as Facebook connect.

Given enough (say 300) distinct face images for each behavioral patterninstance, an image-based classifier is trained to predict which patterna certain person belongs to, based on its face image alone. Then,whenever certain online strategies are developed per each behavioralpattern of a group of patterns, and one or more face images of a personare available, that person can be associated with the correct behavioralpattern, based on its face image alone.

Specific Facial Structure Classifier

In a specific embodiment of the present invention, a classifier for asingle facial structure is trained from examples. One such structure canbe the eyebrows distance from the eyes. In a similar manner, classifierscan be trained for multiple structures:

-   -   Face width: wide/balanced/narrow    -   Lower lip: full/medium/thin    -   Sharp/bulbous noise    -   Nose to upper lip distance    -   Eye orientation (as shown in FIG. 8).

This is done by collecting examples for the different variations of eachsuch face structure and applying the descriptor extraction and machinelearning/classifier training process according to the present invention.One may choose to train the classifier on the entire face, or to croponly the specific part of the face train the classifier on that partonly.

Single-Trait Classifiers

According to a specific embodiment of the present invention, a singletrait classifier is constructed from examples. This may be done in oneor more of the following schemes:

-   -   Use metadata to group people that share a clear trait, and then        train a classifier to predict that group membership and finally        label the classifier by that clear trait. For example one may        assume that a certain group (say Poker players) comprises risk        takers, another group (say nurses) comprises care-givers and        another group (say MMA fighters) comprises aggressive persons.        Thus one may obtain specific trait classifiers from group        metadata (profession, hobby, etc.)    -   Use crowd source in social networks or services such as Amazon        Mechanical Turk to label faces with certain traits (for example        reliability) based on the assumption that if a certain        personality trait or capability can be related to face        appearance, that a classifier trained based on such crowd source        will be able to predict such trait or capability.    -   Run multiple face structure classifiers, trained as described        above on certain groups of people, tagged by        profession/hobby/online behavioral metadata and rank said        classifiers with their relative ability to predict such metadata        correctly from face images only. For example, if near eyebrows        are frequent in successful sales persons and MMA fighters, than        one may deduce that near eyebrows indicate tendency to        close-range emotional or physical contact.

Multi-Trait Personal Characterization

According to one embodiment, a user profile of interest comprisesmultiple traits. These traits may be non-weighted (or equally weighted).In a different embodiment the user profile comprises weighted traitswhere the weight signifies the absolute/relative significance of thetrait in the context of a specific profile. In some embodiments, certaintrait weights may be negative.

As a specific example, consider a user profile which is a buyer profile.Characterizing buyers is of great importance, in marketing and inparticular in online marketing and sales applications. Let the buyerprofile may be one of the following: Early Adopter, Economical, Me Tooor Sensitive.

In a certain embodiment it is desired to select one primary profile foreach online buyer. In another embodiment better conversion may beobtained from assigning a secondary profile to a user, and so on. Forexample, a buyer may be assigned an “Economical” primary type and a “MeToo” secondary type.

Social science as well as behavioral and marketing research may beuseful in identifying a set of personal traits associated with buyertypes. Refer to FIG. 12 for a set of buyer-related traits 1210.

Based on such sources of information, as well as certainheuristic/common sense, one may specify certain buyer types by aselection of traits, target or ideal values for these traits,designating the trait as Dominant, etc. and assigning a certain weightto each trait. The result of weighting is a total score valuerepresentative of the probability that the buyer is indeed of thatspecific type.

FIG. 13 depicts how such a score can be computed from biometricinformation such as face images, according to an embodiment of thepresent invention. According to the present invention a classifier isconstructed or trained per each trait of relevance and then duringruntime, used to produce a trait value—for example an integer in therange [1,5] (as indicated by numerals 1031-1034).

The individual trait values (1031-1034) are then combined to generate acomposite score for the specific buyer type as indicated by numeral1040. A common form for weighting and combining is a linear, weightedsum as depicted in FIG. 13. Alternatively one can use decision trees.

The framework depicted in FIG. 13 may be extended to include auxiliaryinformation from other source of data. According to the presentinvention Auxiliary information is passed to trait classified #M+1 to#N, weighted by the respective weight and then added to the summation.

Now that we have generated buyer-type scores for each buyer type, we mayselect the appropriate buyer type by running a series of buyer-typeclassifier (as indicated by Buyer Type A, Buyer Type B, Buyer Type C andBuyer Type D in FIG. 14)—one for each buyer type and then selecting thebuyer type yielding the largest score, as indicated by numeral 1041 inFIG. 14.

Going back to FIG. 13, assigning weights heuristically as originallysuggested by the present invention has the advantage of building userprofiles from general social science knowledge, without any ground truththat can be used for training. This may be useful in the field of HumanResources (HR) where we may want to seek for best candidates for a newposition, and often we do not have enough positive and negative examplesof people who have assumed that role in the past.

In other application, we may have access to tagged user data and we mayuse it to assign weights to the different traits, thus improving ourclassification accuracy based on past performance.

Considering the buyer type, one may have access to images of users thatmay be considered early adopters, based on their past onlinebehavior—such as searching for/purchasing gadgets and new productsonline.

As another example, one may have access to images of users that may beconsidered “Me Too”, based on their past online behavior—such asextensively browsing user reviews, searching for/purchasing productsthat scored high in points and number of reviews, etc.

Given the output of the trait classifiers for these user groups, theweights may be adjusted automatically, using known methods of supervisedtraining.

Given a set of training examples, each marked as belonging to one of twocategories, an SVM (Support Vector Machine) training algorithm builds amodel that assigns new examples into one category or the other. An SVMmodel is a representation of the examples as points in space, mapped sothat the examples of the separate categories are divided by a clear gapthat is as wide as possible. New examples are then mapped into that samespace and predicted to belong to a category based on which side of theclassifier gap they fall on.

For the case of buyer types, as for many embodiments according to thepresent invention, there are multiple types and Multiclass SVM isrequired. A common solution to the multiclass problem is to reduce it tomultiple binary classification problems by building binary classifierswhich distinguish between (i) one of the labels and the rest(one-versus-all) or (ii) between every pair of classes (one-versus-one).

Classification of new instances for the one-versus-all case is done by awinner-takes-all strategy, in which the classifier with the highestoutput function assigns the class (it is important that the outputfunctions be calibrated to produce comparable scores).

For the one-versus-one approach, classification is done by a max-winsvoting strategy, in which every classifier assigns the instance to oneof the two classes, then the vote for the assigned class is increased byone vote, and finally the class with the most votes determines theinstance classification.

In the case that we have a large collection of untagged user imageswhich we believe represent diverse types (for example buyer types), onemay apply unsupervised learning schemes such as k-means in a space oftrait values.

HR Application

The present invention teaches how to match a candidate to a job/functionbased on the candidate's face images. According to the presentinvention, the candidate's personality characteristics can be derivedfrom the candidate's face image in a fully automated manner.

Social sciences and HR practices are used to list/name the personalitytraits of relevance, and also assign weights to the trait in order toderive at least one multi-trait score which will be used to rankcandidates.

The weights are assigned per specific job based on social science and HRagency expertise. For example if the job is R&D team leader, theninterpersonal & management capabilities as well as responsibility willbe weighed more than creativity and verbal communications. Such anapproach is suitable to the structure depicted in FIG. 13 with theweight values derived manually.

Alternatively, one may collect numerous face images of successful R&Dteam leaders for example based on their position in leading companies,number of patents granted, and LinkedIn endorsements and so on. Thenusing the traits as descriptors, a learning algorithm such as SVM mayadjust the weights optimally.

3D Face Analysis

Deriving personality traits from 2D images has the advantage ofavailability. Practically all face images available online in socialnetworks such as Facebook and LinkedIn, in picture sharing application,etc. are 2D. Furthermore, almost all face capturing devices such aswebcams, laptop/tablet camera, wearable devices such as Google Glass,etc. are 2D as well.

Recent advances in sensor technologies make the acquisition of depthmaps applicable to face images as well. Depth sensing techniques includestereoscopic 3D (S3D), structured light sensing such as original Kinectsensor by PrimeSense and Time-Of-Flight techniques as employed NewKinect camera of Xbox One gaming console.

Having a depth map, in addition to the RGB image, facilitates facedetection, pose detection, face alignment and face recognition step asdescribed in the present invention. Furthermore, a single/few RGBZimages may be used to detect personality traits which usually require aprofile image such as nose and chin-related traits.

Therefore, the specific embodiments of the present invention cover notonly 2D images (color images, luminance images, etc.) but also depthimages and combined RGBZ images.

In a specific embodiment, the information captured from the depth sensoris represented in the form of a depth map and in the form of a luminancemap. Face pose is derived from the depth map and used to align both thedepth and luminance representation of the face.

Trait classification is performed independently on each representation(following a process of learning on a large set of such aligned maps).The results are then combined to a single value of the personalitytrait, as a weighted average with either preset weights or dynamicallyadjusted weights based on the estimated precision of each value.

According to another embodiment, a classifier such as SVM is trained ona combined set of features extracted from both the depth and theintensity image.

In another embodiment, the training process which has to rely on a largenumber of images is still performed in 2D, which the prediction processuses RGBZ images for better normalization and alignment, and alsoexploit “profile” image features from frontal faces. When collections offace depth images become available, the training process according tothe present invention can be extended to such images as well.

Search Engine Applications

The ability to tag human faces with metadata which is directly(“adventurous”) or indirectly (“researcher”) to personality traits andcapabilities, can be integrated with any search engine technology tofurther narrow search results/improve the accuracy of search. Such tagsmay be computed offline and indexed to facilitate search using prior arttechniques of textual search.

Alternatively, personality traits may be computed online as the finalstep in the query execution. The faster textual elements of the queryare executed first, resulting in a short list of search results. Then,face images are retrieved for each of the top textual searchresults/matches and undergo the processes depicted in FIGS. 1, 2 and 3.

Finally, the trait-based search filters/criteria are applied to thepersonality/capability/behavior descriptors derived from said faceimage.

The terms, “for example”, “e.g.”, “optionally”, as used hereinabove, areintended to be used to introduce non-limiting examples. While certainreferences are made to certain example system components or services,other components and services can be used as well and/or the examplecomponents can be combined into fewer components and/or divided intofurther components.

The aforementioned program modules include routines, programs,components, data structures, and other types of structures that performparticular tasks or implement particular abstract data types forpredicting personality traits, capabilities and suggested interactionsfrom face images. moreover, those skilled in the art will appreciatethat the invention may be practiced with plurality of computer systemconfigurations, including hand-held devices, multiprocessor systems,microprocessor-based or programmable consumer electronics,minicomputers, mainframe computers, and the like. The invention may alsobe practiced in distributed computing environments where tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules may be located in both local and remote memory storage devices.

The aforementioned embodiments of the invention may be implemented as acomputer process (method), a computing system, or as an article ofmanufacture, such as a computer program product or computer readablemedia. The computer program product may be a computer storage mediareadable by a computer system and encoding a computer program ofinstructions for executing a computer process. The computer programproduct may also be a propagated signal on a carrier readable by acomputing system and encoding a computer program of instructions forexecuting a computer process.

The functions described hereinabove may be performed by executable codeand instructions stored in computer readable medium and running on oneor more processor-based systems. However, state machines, and/orhardwired electronic circuits can also be utilized. Further, withrespect to the example processes described herein, not all the processstates need to be reached, nor do the states have to be performed in theillustrated order. Further, certain process states that are illustratedas being serially performed can be performed in parallel.

All the above description and examples have been given for the purposeof illustration and are not intended to limit the invention in any way.Many different mechanisms, methods of analysis, electronic and logicalelements can be employed, all without exceeding the scope of theinvention.

What is claimed is:
 1. A method of adapting computerized interactionsaccording to an analysis of at least one facial image, comprising:providing at least one facial image imaging a face of an individual;applying at least one image-based classifier on the at least one facialimage for identifying at least one trait value of at least one humanpersonality trait of said individual, the at least one image-basedclassifier is generated by applying at last one machine learningalgorithm on a plurality of training images imaging faces of multipleindividuals; adapting a computerized interaction with the individualaccording to a combination of the at least one trait value anddemographic data relating to the individual.
 2. The method of claim 1,wherein the at least one trait valuecomprises a at least one trait valueof a plurality of human personality traits.
 3. The method of claim 1,wherein adapting comprising modifying an avatar according to thecombination the at least one trait value and demographic data relatingto the individual.
 4. The method of claim 1, wherein adapting comprisingadapting a member of a group consisting of: a video chat, a conferencecall, a wearable computer action, a client relationship management (CRM)action, a set-top box action, and a gaming console action.
 5. A methodaccording to claim 1, wherein the at least one human personality traitcomprising at least one personal capability and at least one humanbehavior.
 6. A method according to claim 1, wherein the adapting isperformed according to the combination and metadata associated with theindividual, the metadata describes a human personality trait selectedfrom a group consisting of: a profession, an online behavior,endorsements fror social network, a real-world behavior, and apersonality profile.
 7. A method according to claim 1, furthercomprising using a search engine for searching for a plurality ofadditional images of the individual and analyzing said plurality ofadditional images to enhance an accuracy of the at least one traitvalue.
 8. A method according to claim 1, further comprising presentingthe at least one trait value on a screen.
 9. A method according to claim1, further comprising analyzing audio signals captured during thecomputerized interaction for further adapting the computerizedinteraction.
 10. A method according to claim 1, further comprisinganalyzing text input captured during the computerized interaction forfurther adapting the computerized interaction.
 11. A method according toclaim 1, wherein the adapting comprises presenting communicationrecommendation during the computerized interaction based on thecombination of the at least one trait value and demographic datarelating to the individual.
 12. A method according to claim 1, whereinthe adapting comprises adapting an advertisement presented during thecomputerized interaction based on the combination of the at least onetrait value and demographic data relating to the individual.
 13. Themethod of claim 1, further comprising converting each of the pluralityof training images into a normalized representation representing acommon frontal pose.
 14. The method of claim 1, further comprisingextracting a plurality of facial key points of the individual, whereinthe at least one image-based classifier is applied on the plurality offacial key points.
 15. The method of claim 14, wherein the plurality offacial key points comprising at least some of the following: eyes, eyecorners, eyebrows, nose top, mouth, and chin.
 16. A system of adaptingcomputerized interactions according to an analysis of at least onefacial image, comprising: a camera adapted to capture at least onefacial image imaging a face of an individual; and a processor adaptedto: apply at least one image-based classifier on the at least one facialimage for identifying at least one trait value of at least one humanpersonality trait of said individual, the at least one image-basedclassifier is generated by applying at last one machine learningalgorithm on a plurality of training images of multiple individuals, andadapt a computerized interaction with the individual according to acombination of the at least one trait value and demographic datarelating to the individual.